Computers and other host devices are often connected with one or more storage devices via a storage area network (or “SAN”). It is well known to establish such networks using the Fibre Channel protocol, which is well suited for transmitting storage data over networks due to its ability to carry different networking and channel protocols. For instance, it is possible to transmit channel protocols such as SCSI (Small Computer System Interface) over Fibre Channel links.
The Fibre Channel communications protocol is generally considered to involve five independent layers (Fibre Channel-0, Fibre Channel-1, Fibre Channel-2, Fibre Channel-3, and Fibre Channel-4. Fibre Channel-0 covers the physical characteristics that allow communications over a variety of physical media, such as copper wiring or fiber optics. The Fibre Channel-1 layer defines the 8B/10B transmission coding developed by Widmer and Franasak of IBM. The Fibre Channel-2 layer defines the Fibre Channel framing and signaling protocol. Fibre Channel-3 defines various link service functions. The Fibre Channel-4 level defines the implementation of the different protocols (such as SCSI) that can be carried over the Fibre Channel connection.
The Fibre Channel-2 layer defines the Fibre Channel frame 10 as shown in FIG. 1. A frame 10 always starts with an ordered set (a specially defined four byte grouping that cannot be mistaken for data) that is known as the start-of-frame delimiter 12. Following this start-of-frame ordered set 12 is a 24-byte frame header 14 that contains basic information about the frame 10, including the type of data being transmitted, a sequence ID, a sequence count indicating the position of frame 10 in the sequence, and an the exchange ID identifying the exchange 40 to which the sequence belongs. After the frame header 14 can be up to 2112 bytes of data payload 16, which contains the actual data being transmitted. In some circumstances, optional headers 18 of up to 128 bytes are included in data payload 16 with along with 1984 bytes of actual data 20. Following the data payload 16 is the frame CRC value 22 and the end of frame ordered set 24. The Fibre Channel-2 layer also defines a Fibre Channel sequence as one or more related frames 10 flowing in the same direction on a link and a Fibre Channel exchange as one or more sequences used to complete a single communication operation.
Fibre Channel based storage area networks are generally limited to campus-sized environments as a result of Fibre Channel's 10 kilometer distance limitation. Recent technology developments have been proposed that would allow these localized SANs to be connected together over Internet protocol (IP) networks. An organization known as the Internet Engineering Task Force (IETF) has helped to develop the specifications for one such technology. This technology, known as Fibre Channel over IP (FCIP or Fibre Channel/IP), enables the tunneling of Fibre Channel data between SANs over IP networks. The FCIP technology relies upon the transport layer (such as TCP) for congestion control and management and relies upon both the transport layer and the Fibre Channel protocol for data error and data loss recovery.
The FCIP specification tunnels Fibre Channel data by embedding each Fibre Channel frame 10 within a single IP datagram. This one-to-one relationship is possible because the maximum size of an encapsulated Fibre Channel frame 10 will not exceed 2148 bytes, while an IP datagram (according to IPv4 as defined in RFC 791) can hold 65535 8-bit bytes (including the 20 bytes that are required for the IP header). Thus, after adding the transport layer and IP headers, the FCIP protocol will turn a stream of maximum-length Fibre Channel frames 10 into a series of 2168 byte IP datagrams. Even if the 40-byte optional IP header is utilized, each IP datagram will be only 2208 bytes in length and hence much smaller than the maximum size allowed for IP datagrams.
Once the IP datagrams are created, they must pass over a physical network utilizing a data link protocol such as ATM, Frame Relay, Ethernet, or IEEE 802.3. In the typical environment, the IP datagrams are divided into Ethernet frames having maximum size 1500 bytes or IEEE 802.3 frames having a maximum size of 1492 bytes. This maximum transmission size for a network is known as the maximum transmission unit or MTU for that network. Datagrams that are larger than the MTU for a network must be divided into MTU sized frames in a process known as fragmentation. Thus, if FCIP creates an IP datagram having a maximum sized Fibre Channel frame 10, this 2168 byte IP datagram will be divided into two separate Ethernet frames to pass over an Ethernet network. One Ethernet frame 60 will contain approximately 1.5 KB of data from the Fibre Channel frame 10 and the other Ethernet frame 60 will contain approximately 0.5 KB of data. Unfortunately, the frame handling overhead required to send each Ethernet frame 60 to its final destination is generally the same even though one of the frames is more than half empty (although the actual transmission time for a half-empty frame is less than the transmission time for a full frame). Thus, the utilization of half-empty Ethernet frames by the FCIP standard reduces the efficiency of the FCIP connection.
FIG. 2 shows the relationship using FCIP between Fibre Channel frames 10, Internet protocol datagrams 40 and data link level frames, such as Ethernet frames 60. The Fibre Channel frames 10 are each embedded into a separate IP datagram 40. This datagram 40 consists of headers 42 created by the transport layer (such as, but not exclusively, TCP) and the Internet protocol in combination, as well as the embedded Fibre Channel frame 10 itself. The IP datagram 40 is then subject to fragmentation when sent over the physical network interface. In FIG. 2, the IP datagram 40 is larger than the MTU of an Ethernet network, and is therefore fragmented into two Ethernet frames 60. The multiple Ethernet frames 60 are then communicated over the physical network and later reassembled into the original IP datagram 40 at the receiving end.
The inefficiency of using half-empty Ethernet frames 60 can significantly affect network performance, especially in data storage applications where a Fibre Channel sequence includes multiple Fibre Channel frames 10 that contain the maximum 2 KB of data. This situation is shown in FIG. 3, where three consecutive Fibre Channel frames 10 of 2 KB are to be transmitted over an Internet protocol network. Each Fibre Channel frame 10 is embedded into a separate IP datagram 40, each of which in turn is fragmented into two Ethernet frames 60. Hence, a total of six Ethernet frames 60 are required to transmit the 6 KB of data found in the three Fibre Channel frames 10.
What is needed is a way to implement a frame-based protocol such as Fibre Channel over a IP or similar network that fully utilizes the available IP bandwidth.